The invention relates generally to the use of structured patterns of light for motion tracking of objects. More specifically, the invention relates to a system and method where projected structured patterns of light based on m-sequences are used in combination with an optical sensor to locate and track an object.
Motion tracking is a fundamental component of almost all interactive systems. The rapidly growing area of embodied and gestural interaction underlines the continuing importance of accurate, fast, and affordable motion tracking technology. Numerous approaches have been developed to sense and track motion using mechanical, inertial, acoustic, magnetic, and radio-based sensing techniques.
In one example of an existing tracking system, natural features in a scene can be tracked using standard cameras to recover three-dimensional information from a series of two-dimensional images, for example, by using Structure from Motion (SfM). When combined with Simultaneous Localization and Mapping (SLAM), camera pose can be estimated within a dynamically constructed map of the environment. Although powerful, natural feature tracking techniques are computationally expensive and challenging to implement in embedded systems and mobile devices.
In another example, marker-based motion tracking systems instrument objects and the environment with physical markers to aid the tracking process. A camera is used to detect the markers in the scene and estimate the location and, in some systems, the orientation of the marker. Marker-based systems can utilize printed two-dimensional barcodes, retro-reflective or light-emitting points, hidden patterns or printed patterns. While some of these systems offer low latency, high precision tracking, commercial marker-based systems typically cost thousands of dollars.
With projected marker tracking, marker patterns are dynamically projected onto surfaces in the environment for use with handheld devices, interactive tabletops, and augmented displays. The projected markers can be identified and tracked using standard marker-based tracking techniques. These systems typically hide the obtrusive appearance of the projected markers using near-infrared projection or temporal multiplexing. Latency can be an issue with these types of systems because a camera is used to sense the projected marker.
Structured light tracking is another example where projecting geometric patterns on the environment, i.e. structured light, allows a camera to infer information about the structure and properties of the environment. A number of structured light schemes have been developed for three-dimension object digitization and recognition including M-arrays, de Bruijn sequences, bilinear de Bruijn sequences, time-multiplexed grey codes, and others. Structured light has been used for interaction to simultaneously capture and display on objects and people, to localize mobile devices, and in a range of interaction scenarios using structured light depth cameras. As structured light systems typically use a camera-projector pair, latency is a common and non-trivial issue.
In yet another example, with projector/sensor tracking systems, projecting light directly onto an optical sensor enables spatial information to be directly communicated between projector and sensor. The dominant approach has been time-multiplexed projection. For example, in one system of this type, spatial information is temporally communicated to photodiodes embedded in objects using LED clusters. In another system, a specially-modified projector is used to project temporal grey code pattern within a monochrome display using an imperceptible change in modulation; sensors within the projection field then read the grey code patterns. With a gray pattern system, the projection is inherently time-multiplexed. In another one of these systems, the visible output of a projector is modulated to transmit time-multiplexed codes onto sensors within the projection field, which activate interactive functions upon detection of an appropriate code.
While existing tracking systems can offer precise tracking in some instances, they suffer from high cost, high latency, or complex designs. Therefore a need exists for an inexpensive motion tracking system that offers low latency and high precision in a relatively simple system.